What do pictures say? Secrets and Lies through Image Manipulation
Hyo-Kyung Choi, Eun-Jung Choi
Dept. of information Security
Seoul Women’s University
Seoul, Republic of Korea
hyokyung@swu.ac.kr, chej@swu.ac.kr
Abstract
When creating a manipulated image, people can modify an image easily using many different image editing tools, leading to quickly create unreliable data which is the source of fake information. Furthermore, there are different degrees of dangers of a manipulation, depending on whether someone deliberately changes the image. That is, creating improper outcome is worse than making a previous picture look better without any intention. Therefore, to trust image information, image manipulation for inappropriate purposes should no longer be increased. Image manipulation techniques used for political instigation, for libel, and for demeaning certain person cause confusion in society and reduce reliability in image technology. Accordingly, in this paper, we introduce image manipulation techniques to understand such deceptions and the abuse cases for warning the use of image manipulation, leading to suggest needs for advanced techniques detecting image manipulation.
Keywords-Image Manipulation; Abuse; Lies; Dangers; Negative Side; Instigation; Libel
1. Introduction
When people read, they understand the context of the sentence rather than read it as it is, and infer the meaning behind it. Over the last few of decades, however, most people had not tried to find a hidden message in pictures, as they would read. This is because in the past, it required considerable effort and expertise to take pictures, and editing techniques had not yet been developed, people did not think there would be any hidden meaning in the pictures. They thought that pictures could only contain facts. But, there has been a growing suspicion of the meanings of photos in recent years because of the secrets and lies of the vast amount of image data available through the Internet. This phenomenon is closely related to the increase in image manipulation techniques.
In modern times, advanced image editing techniques became so widely available and this phenomenon has made it easier to edit images for non-photo professionals, and easier image editing techniques have increased the trick called image manipulation which distort facts. People are confused with real and fake information because many such manipulations are spreading quickly through the network, even though these manipulations are not 'white lies'. Hence, if you do not doubt whether pictures are true when you look at photos in modern world, you will trust fake information, your judgment will be blurred, and sometimes you will join in undermining someone’s reputation. Consequently, image manipulation expedites the creation of negative sides of photography technology, thus, this manipulation can be very risky proposition.
In this vein, trusting the fascinating photography technology requires the ability to doubt manipulated pictures. This includes not only detecting image manipulation technically, but also recognizing the negative aspects of image manipulation and being wary of its dangers. In other words, image manipulation techniques are very sophisticated like any other camouflage techniques, so people must always respond to this sensitively.
In this paper, we provide the reasons why you should seriously consider these pending issues of image manipulation, and introduce an approach that helps these issues: we refer to specific types of image manipulation techniques to understand overall image manipulation techniques, and present the inappropriate cases to illustrate risks of abuse of manipulations.
The structure of the rest of this paper is as follows. In section 2, we describe image manipulation techniques for understanding image manipulation. The descriptions of image manipulation techniques consist of editing, generating and hiding. In section 3, we introduce far reaching and long lasting abuse cases of image manipulation technology targeting politicians and celebrities. In section 4, we reiterate our findings with attesting to the value of our approach. Finally, in section 5, we discuss plans for future work.
2. Image Manipulation Techniques
2.1. Editing
In this section, we give a brief overview of image manipulation techniques. Image manipulation techniques are varied and abundant. In order to introduce how to manipulate images, we divided them into editing techniques, artificial intelligence techniques, and hiding techniques. The first category is editing. Various editing methods to make photos look better than the original image result in image manipulation techniques. These techniques range from very simple outline modification to graphical animation to create vividness effect. You can use the shadows in the image to give a three-dimensional effect, delete the background, insert a new background, give perspective to the image by modifying the proportion, express colorful colors through color blending, and combine and composite two or more images, or you can cut out only the specific parts you want to emphasize, create a symmetry, invert, extract a part, cut and rotate, and so on. If you use these editing techniques well, this would contribute to create an artistic work. Especially many artists use editing technology to express their personality in images. However, this technology can be unlimitedly sophisticated depending on who uses it, so when exploited, it can create fake information that is difficult to distinguish.
2.2. Generating and Hiding
The second category is generating and hiding. These techniques are precisely data manipulation, the narrower range than of image manipulation. First of all, generating is a technique that uses the artificial intelligence technology to generate a desired image based on the original image. With this technology, you can create the image you want without directly manipulating the original image. The images generated by artificial intelligence are very powerful. If you have done the coding for creating the image you are implementing, because it is literally automated, to make images more versatile is only a matter of time and the performance of computer. That is, manipulating images with artificial intelligence is more dangerous when you have a malicious purpose. The speed is too fast, and sophistication has already surpassed manipulation skills that human can deal with. Second, Hiding is the sequence of actions that hides something in the raw data. It is also a technique for hiding and distributing malicious code in images. Although it is not a technique for manipulating the appearance of an image, the fact that it can hide information in raw data means it is possible to use raw data to detect image manipulation. In other words, this technology is more useful for image manipulation detection than to know the danger of image manipulation.
3. Abuse Cases
The various image manipulation techniques mentioned above are not always used with good intentions. In Berlin on September 10, 2015 – Syrian refugee Anas Modamani snapped a selfie with German Chancellor Angela Merkel. However, the photo has appeared in numerous false stories on social media. He was falsely identified as one of the refugees who set fire to a sleeping homeless man in Berlin and he was also falsely identified as Najim Laachraoui, one of the terrorists behind the Brussels bombings in March 2016 [1]. The misinformation spread by his manipulated photographs led to criticism of Merkel’s refugee policy, and Modamani filed a lawsuit against Facebook, which allowed to spread manipulated photographs without sanctions, but the Wuerzburg court ruled that although the posts were ‘undisputable defamation’, Facebook had not created the slanderous content itself, and so was not required to actively seek and delete the posts [2]. This case set a precedent for demonstrating that the purpose of a manipulated photo can cause agitation for a particular policy.
Fig. 1 Manipulated Modamani image [3].
Another issue of manipulated images comes from the act of abusing fame. Fabrication of celebrity porn pics is nothing new anymore. In late 2017, a user on Reddit named Deepfakes started applying deep learning to fabricate fake videos of celebrities [4]. Applying machine learning technique which learns patterns from the data and applies them to new data, the pornographic images began to spread on the Internet with manipulated images of celebrities such as Scarlett Johansson, Taylor Swift, and Emma Watson. The problem is that it is easy to use such techniques applied to this porn. Anybody who even has little knowledge of machine learning algorithms can easily create fake porn using free open source code. This is an ethical issue, but it is also a test of human cognitive ability [5]. Also, the emergence of these manipulations is likewise exploited by using fames of politicians. In April 2018, a fake video that former US President Barack Obama is telling US President Trump is a dipshit was released on the Internet. In these cases, artificial intelligence technique was used, and, it is clear that it still synthesizes faces of celebrities and spreads them maliciously. Celebrities make curious, and their manipulated photos spread faster than any other pictures. Just because they have a reputation, they suffer from their own fabricated pictures that wander around the Internet. When this phenomenon becomes generalized, the reliability of the real image may deteriorate later. As a result, there are concerns that manipulations could cause serious problems of the international community, within a year or two [6].
Fig. 2 Fake video of Barack Obama [7].
The last case is an incident that happened in Korea. In Korea, cybercrime that degrades a specific person is a very serious problem, and this is usually done by defaming his or her reputation with manipulated images. The most famous case is image manipulation done by a website called Ilbe. Members of the website Ilgan Best Jeojangso (Ilbe, storage space for the daily best) have strong anti-North Korean views and are at the center of controversial remarks snubbing former Korean President Roh Moo-hyun and the May 18 pro-democracy movement in Gwangju [8]. They have been constantly manipulating and distributing images with wrong purposes. In Korea on May 18, 2018 – KBS (Korean Broadcasting System), the Korean public service broadcasters, mistakenly broadcasted Ilbe image which is manipulated by that website. Most viewers were angry at the manipulated image used to ridicule the former president. As the controversy continued, the production team of the program apologized to viewers on 19th that it was an obvious mistake [9].
Fig. 3 Differences between the original logo image and the manipulated logo image [10].
Unfortunately, there is not enough technology or legal resource to directly manage all these issues. In any part of the world, even if someone is suffered by image manipulation, they cannot severely punish or receive suitable prevention. For this reason, we must face the negative sides of image manipulation. We need more powerful techniques to detect image manipulation and more specific legal resources. In addition, if the development of image technology will be encouraged for good purposes and the wrong intentions are broken, we could be able to experience better technological progress.
4. Conclusion
In this paper, we introduced how images tell the false and how the aftermath of the lies of image manipulation can have a major impact on society. In some ways, techniques of manipulating images have an attractive artistic value and people can use the techniques for good goals. However, the risk of image manipulation is so large and powerful that it cannot justify this enticing temptation. The information on the modern society is ‘sharing’, and it always expects basic honesty. That is, good information requires reliable data. While you may think that the above-mentioned deceptions are not relevant to you because these are confined to political issues and celebrities, the abusing modern computing technology is at any time and from anywhere, targeting everyone.
In conclusion, we urge the urgent development of technology of detecting image manipulation. We are living in an age where we are exposed to a remarkable array of visual imagery [11]. While you hesitate to doubt, unreliable data is being created with great effort even at this moment. That is why you should always be vigilant about image manipulation. Which means, there is a concomitant need for developing techniques to distinguish the original images from the altered ones [12]. When developing techniques to detect image manipulation, it must be more sophisticated and more accurate than existing technology. This is because, that image manipulation detection is different from traditional semantic object detection because it pays more attention to tampering artifacts than to image content [13].
5. Future work
Future work concerns new proposals to detect image manipulation. There are some ideas about the detection. Obviously, these ideas will answer the question: Can human brain truly detect manipulated images? Firstly, it could be interesting to consider the use of artificial intelligence technology. There are already too many manipulated images on the Internet, and this has been beyond the human processing speed. The problem of speed due to changes in modern technology can always be addressed only by technology, and AI could be an alternative. Secondly, we could achieve detection by analyzing the binary code. By analyzing the binary code, we can observe information that is not visible from the exterior. We will get detailed information about image manipulation, including whether it is manipulated, the process, the date, etc. This will definitely help detection. To sum up, we will use these ideas to develop detection techniques that can eliminate the risk of image manipulation and to overcome human limitations.
Acknowledgment
This research was supported by the MISP (Ministry of Science, ICT & Future Planning), Korea, under the National program for Excellence in SW (2016-0-00022) supervised by the IITP (Institute for Information & communications Technology Promotion).
References
[1] https://www.aljazeera.com/indepth/features/2017/02/selfie-merkel-changed-syrian-refugee-life-170218115515785.html
[2] https://www.bbc.com/news/world-europe-39197195
[3] https://www.newss.co/2017/02/06/syrian-refugee-sues-facebook-over-defamatory-posts/
[4] https://medium.com/@jonathan_hui/how-deep-learning-fakes-videos-deepfakes-and-how-to-detect-it-c0b50fbf7cb9
[5] https://www.bbc.com/korean/news-42942152
[6] https://news.joins.com/article/22766974
[7] https://www.youtube.com/watch?v=cQ54GDm1eL0
[8] http://english.khan.co.kr/khan_art_view.html?artid=201305221617407&code=710100
[9] https://news.joins.com/article/22638436
[10] http://news.zum.com/articles/45225176?cm=popular
[11] Hany Farid, "A survey of image forgery detection," IEEE Signal Process. Mag., vol. 2, no. 26, pp. 16-25, 2009.
[12] Sevinç Bayramİsmail AvcıbaşBülent SankurNasir Memon, “Image manipulation detection with Binary Similarity Measures,” Proc. European Signal Processing Conf., 2005.
[13] Peng Zhou, Xintong Han, Vlad I. Morariu, Larry S. Davis, “Learning Rich Features for Image Manipulation Detection,” arXiv:1805.04953, 13 May 2018.